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ABSTRACT 

Both interview and direct observation data were 
collected via on-site visits with the training directors at 22 U.S. 
training centers which had been identified as the best training 
facilities in order to examine why the instructional methods, 
techniques, and media changes in use had been tried and adopted, and 
to identify the trends these methods represent. The organizations 
studied included computer manufacturers, universities, military 
contractors, military academies, an airline, and a simulator 
developer. It was found that most of the organizations perform a 
needs assessment analysis to aid in choosing instructional methods 
and media, and that there is a trend toward individualized 
instruction and real environment simulations. While the organizations 
were researching computer-assisted instruction, interactive video, 
instructional television, and distance learning as alternatives to 
current training programs, only instructional television was in 
widespread use. Many successful programs were under the direct 
supervision of a high level administrator and funded at the corporate 
or division level; more innovation and creative activity seemed to 
occur if this was the case. Almost all class instruction was 
evaluated in some manner; however, materials and overall program 
evaluation was not consistently performed. Specialized according to 
instructional objectives, clients , and security, physical facilities 
included multimedia classrooms, most of which included equipment to 
project overhead transparencies and display videotapes and/or live 
television presentations. (13 references) (DB) 
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What are the best training organizations in the United States 
currently doing to maintain their excellence? To determine the ap- 
plicability of state-of-the-art instructional technology and teaching 
strategies to Navy classroom training, the Navy Personnel and Research 
and Development Center supported a study to evaluate these training 
programs. On site visits with the training directors at identified 
training centers were arranged so that both interview and direct 
observation data could be collected. 

This study examined why the instructional methods, techniques and 
media changes were tried and adopted. The trends these methods repre- 
sent are of special interest. The identities of many organizations 
are not identified here to honor the conditions under which the visits 
were arranged. Many organizations were not concerned with the pro- 
prietary nature of the information given in their interviews and those 
organizations and individuals are identified where needed to illus- 
trate specific points or generalizations. 

Twenty-two site visits (see Table 1) and interviews with key 
training directors occurred beginning in the Summer of 1987. The 
programs, all located in the United States, were identified as the 



t"* ' best training facilities, by a panel of experts. The twenty-two 
| organizations in the study included a wide-range of organizations: two 
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major computer manufacturers, a computer systems developer with an 
experimental school, a CAD/ CAM training organization, an aerospace 
training organization, three universities, a television-based college 
degree granting organization, two military ISD-oriented training 
contractors, two military academies, three military training "com- 
mands," a government staff college, a telephone-oriented training 
facility, an airline, an A-V manufacturing organization that does 
extensive internal and external training, an AI software developer, 
and a training simulator developer. 

Trends in Training 

Analysis of Training-Related Problems 

Front-end analysis, in the form of a needs assessment, job analy- 
sis, and/or tasks analysis was stressed as a requirement for designing 
and developing instructional materials by all individuals interviewed 
in the study. However, the data suggested that time was infrequently 
taken to perform an extensive analysis. Typically instructors, and 
occasionally subject matter experts (SMEs) , are asked to provide 
copies of their materials (course objectives, outlines, tests and 
handouts) and requested to make suggestions for improvements when 
redesigning or developing existing courses. Training procedures for 
developing entirely new courses and materials involve a more complete 
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analysis than is common otherwise. The analysis for new courses often 
involved gathering data from SMEs, engineers, and from formative 
evaluation data and, occasionally, pilot groups. 

One newly developed program for middle managers is based on 
analysis of data from (a) a questionnaire sent to advisory group 
members, (b) interviews or telephone contacts with individuals in the 
projected positions who were asked what they felt they needed to learn 
in formal course-work, and (c) faculty members/courseware designers 
hired because of their related expertise. No apparent attempt was 
made to sequence courses and classes, nor make hierarchies of objec- 
tives and tasks except on an informal "logical" basis. 

The best training/ educational front-end analysis was found in an 
academic setting. After obtaining "complete" support for a major 
project from an client — in the sense they understand they have a 
problem and that his Center was the right agency to work on the 
problem — the resource center director then spent two-thirds of the 
total project time (typically the Fall and Spring semesters during a 
school year) identifying and specifying the instructional problem, 
determining the objectives and the media requirements. The remaining 
time (typically during the summer semester) he developed the educa- 
tional program and associated materials. Except, perhaps, for the 
unusually long time spent in getting user commitment to the instruc- 
tional development effort and in obtaining the user*s commitment to 
use the resulting program/materials, the analysis portion of their 
instructional development model looked like many military Instruction- 
al Systems Development (ISD) models. 



Design & Delivery Modes 

Most industrial and military training organizations had ISD 
models they generally follow. Complicated presentations were more apt 
to follow the rigorous instructional development procedure required by 
the selected instructional development model. 

Many of the training organizations are moving toward individual- 
ized instruction . The use of interactive video disks, and of comput- 
ers frequently seems to lead these organizations away from totally 
grouped activities. Many of these organizations are refurbishing 
their facilities and eliminating some classrooms by tearing out inte- 
rior walls. The resulting large open spaces permit installing study 
carrels or tables for computerized delivery of individualized instruc- 
tion. 

A few organizations are attempting to design, or create, group- 
oriented "enriched classrooms" (their term) . These classrooms look 
like, and often operate like, a real work environment. These class- 
rooms are really training simulators. These simulators are usually 
"partial-task" simulators that focus on specific aspects of the real 
environment, but do not permit a real-time simulation of the entire 
environment being trained. Organizations that advocate work simula- 
tors feel that transfer to the real world is enhanced if instruction 
is provided in an environment that resemble the real world. One site 
had researched "enriched classrooms" and believed student retention of 
information is greater in these environments. There is ro published 
documentation available on this issue. 

Computer-based simulators , as well as physically simulated work 
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environments, seem to be increasing in number. SIMNET (a tank train- 
ing simulator) , AIRNET (a combat environment) and STEAMER (a power- 
plant simulator) were observed. Some of those involved in these 
computer-based simulators believe that the future of education lies in 
simulating real environments both physically and by computer program- 
ming. They feel that Al-based simulations will result in easy inter- 
facing between learners and the "instructional program" or simulation. 
SIMNET strengths are well documented. U.S. "tankers" who have, 
historically, lost mock battles to German tank crews began winning 
these competitions after U.S. tank crews practiced using SIMNET 
prototypes. The tank crews give highly visible credit to their SIMNET 
training. 

Typical comments included: "We are attempting to leave prescrip- 
tive teaching/ learning (conventional instruction) and go toward 
adaptive learning systems which are simulation-based; where the 
learner can focus on a wide-range of 'relevant' problems." "In the 
future computer-based simulation, using a lot of graphics, will be 
developed using Artificial Intelligence layers to ease the user 
interface by accepting expected or approximated responses as appropri- 
ate responses — approximations thereof. The students should be given 
the problem first, then the challenge (to promote student interest and 
retention) and then the necessary and relevant information or process- 
es." 

Computers and Other Forms of Instructional Technology 

Most of the visited organizations are researching Computer. Asz 
sisted Instruction but no one was actually using it for more than five 
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percent of their instruction. The Air Force Academy has evaluated 
many authoring systems and is the closest (of the visited organiza- 
tions) to making a large-scale commitment to CAI. They plan to use 
computer-based authoring systems to generate CAI materials, to develop 
classroom graphics (which can be projected in the classrooms), and 'to 
permit students who miss lectures to review the computer-based presen- 
tations themselves. The Language Department said that they had se- 
lected an interactive video disk-based authoring system for use in a 
limited number of classes for about 50% of the student "class-contact 
hours." They expect to use the interactive computer assisted instruc- 
tion system mostly for drill and practice exercises. 

Some organizations are adopting interactive video disks, IVD, 
technology for internal training. One computer manufacturing company 
expects that 50% of their field service training (much of it is main- 
tenance training) will be IVD-based while another computer company 
expects that 75% of their internal training will become individualized 
within five years — they expect to use IVDs for a major portion of 
this individualization. They have found that instructor-SMEs located 
in the training centers can supervise and answer questions from stu- 
dents in a wide-range of courses at one time. This instruction is 
considered individualized but group-based since only 40% to 50% of the 
time is spent in IVD training and the rest of the time is spent in 
group-oriented labs and lecture/discussion sessions. 

Instructional television is a workhorse medium of instruction in 
many of these better training organizations. Special TV lighting is 
no longer required given the current cameras which can operate at low 
light levels. 
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Dissemination of ITV courseware via electronic distance ^earning 
procedures is increasing- Many of the corporate and the military 
training centers (e.g., the Logistics courses from Ft. Lee, VA and 
some I.B.M. management courses) either own their own satellites or 
lease time as cost-effective ways of disseminating quality videotaped 
or live instructional presentations. In many cases, feedback was 
encouraged from the receiving classrooms by telephone, and in other- 
cases, by video link. 

In the degree granting arem* of adult education there are few 
institutions which use instructional technology as the primary in- 
structional medium. Most college credits are based on "contact hours" 
instead of content: mastery. Colleges and Universities are norm rather 
than criterion reference oriented. With the exception of the National 
Technological University (NTU) , which distributes eight Masters Degree 
program, few institutions in higher education are regularly using 
television for their "distance education" programs. Robert Diamond, 
at Syracuse University, was typical of other interviewees when he said 
that until there is good ITV software, there will be little call for 
television-based instruction. 

Program Management zjxd Development 

Most of the interviewed training program managers reported to a 
senior Vice-President or other high level administrator in their 
organizations. In addition, the successful training managers had a 
great deal of support from their "line" administrator. Robert Dia- 
mond, for example, insisted as a prerequisite to his taking the 
position at Syracuse, that he have an appointment sufficiently high in 



11 



the administrative structure (Assistant Academic Vice-President) so he 
could interact with the academic Deans and University Directors as a 
peer. 

Another management- related variable which was frequently, but not 
consistently found in this survey, was that instructional development 
activities were funded at the corporate or division level and not by 
the individual using department. Lower level budgeting seems to 
require a lot of paperwork and approvals which result in poor use of 
personnel and resources in training centers. When the funding of the 
centers occurred at the higher level, and the responsibility for J| 
prioritizing and supporting activities was with the training director, 
more innovation and creative activity seems to occur. The managers of 
these centrally funded centers suggested that they were expected to 
use their funds in a cost-effective manner; thus, (a) courses with the 
larger numbers of students, or (b) those which had the worst problems, 
or (c) those requiring less costly fixes were the projects which re- 
ceived the top priority. The money used for billing and associated 
paperwork could be better spent for additional production materials 
and personnel. 

The development of the training materials is often done by a 
team . The instructional development at CADAM was done by training 
"instructional designers" the content so they could design the train- 
ing materials. The actual teaching, however, is usually performed by 
SME personnel; CADAM feels the peer relationship of their instructors 
to the students is very powerful — the students need the. "real world" 
problem solving input that is possible only from experienced individu- 
als. The SME-instructors and the ISD-designers are physically situat- 
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ed togetner in two-person offices; this permits nec-ssary interaction 
and immediate feedback as the materials are developed and as the 
materials are discovered to have problems when they are used. In a 
military training organization involved in developing CAI materials, 
they attempt to locate computer programmers and teach them how to be 
instructional designers who they then team up with subject-matter 
experts. 

Consolidation into a central " instructional services" or. iiA-V 
center . ". not diffusion, of services into "satellite centers" was 
evident. This was due to the expense of the specialized personnel 
(writers, instructional designers, graphics personnel) and the cost of 
specialized hardware (CAD/ CAM and other graphics hardware especially) . 
Despite the current literature which often talks about the development 
cf media centers in multiple locations near the end user, this occur- 
rence was not evident in the centers visited during the past two 
years. Many large corporations had but one media production center on 
a site to service multiple users. In corporations with competing 
divisions, there was occasional duplication of services on a geograph- 
ic site, but this duplication was unusual. Where divisions in a 
corporation seemed to actively compete, a division without a specific 
capability almost always seemed to use "outside" vendors for its A-V 
services even though these services were available in the competing 
division. In at least two organizations that the internal competing 
service center's rates were much higher than those available outside 
the corporation. 

An administrative issue which this study could not resolve 
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involved the actiu 1 adherence of training organizations to current 
copyright law. First, it was not easy to observe the compliance with 
copyright practice. Second, it was difficult to obtain reliable 
statements about copyright practices. 

Evaluation / Revision 

Almost all class instruction was evaluated in some manner While 
the training managers stressed that instructor effectiveness informa- 
tion is used by them and the instructors to evaluate the success of 
the instruction, the instructors often indicated they did not bother 
to even read the evaluations. The information was used by instruc- 
tional designers to revise and improve the instruction. Information 
provided by the instructors indicated that the classroom evaluation 
data was not used very often as a significant part of the instructors 
annual merit review for pay or promotions purposes — despite policy 
statements to the contrary in many organizations. These statements 
were commonly made in both military and in corporate training units. 

Materials and overall program evaluation , either formative or 
summative, was not consistently performed. Formative evaluation of 
materials and the overall program by target audience students, as it 
is being developed, is preformed (it appeared) in less than half the 
development projects. This surprised me, but as one manager said, "It 
sure is hard to evaluate your own materials as you are too ego in- 
volved." Another manager said of contracted materials: "The contractor 
doesn't really want to do too much evaluation because they want to 
complete the materials without a cost-overrun and we often don't do 
too much evaluation ourselves because we want to believe we got a good 
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value tor our training dollar." 



Physical facilities 
One of the newer training organizations identified the types of 

training facilities they required. In summary, they are: 

1) auditorium (for guest lectures and ceremonies) (with rear 
projection capability) ; 

2) large classrooms for up to 50 students (these had built- 
in TV, VCR, overhead projectors, 35 mm slide equipment); 

3) seminar rooms, or "breakout areas," for up to 15 students 
(where much of the instruction would occur in this organiza- 
tion) (with built-in overhead projectors — many breakout 
areas had computers or terminals for each student [for 
problem solving, CAI, work on simulations [if networking is 
available], and electronic messaging [if networked], word 
processing, data base management, and "developing positive 
student attitudes toward the course and computers"); 

4) individual learner areas ("offices" or carrels where 
students can store materials, go to work on individual 
problems — without going back to their homes or dorms; 

5) other identified training support areas necessary includ- 
ed: dormitory rooms, cafeteria, library, bookstore, TV 
studios, mail room student services (book issue, typewrit- 
ers, student A-V labs to produce materials for student- 
developed presentations, photocopy facilities) and a lounge 
area. All facilities were accessible to the handicapped. 

In addition, the instructors and program administrators had space 

requirements which included: offices for the faculty and staff (3 or 4 

to an office) and storage areas for their instructional media and 

materials. 



Almost all Multi-media classrooms visited during this study 
included equipment to project overhead transparencies and display 
videotapes and/or live television presentations. Classrooms for 
clients and executives often had built-in television projectors 
mounted out of the way on the ceiling or had rear-view display capa- 
bility. Some of the classrooms had equipment to display computer- 
based video displays. A few of the classrc">ms had remote-control 
panels or telephone systems connecting them to the central distribu- 
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tion center so they could pause or start presentations. Nearly a half 
of the classrooms had the ability to load and operate a VCR located in 
the classroom and many had both remote and local television distribu- 
tion capability. Classrooms designed for groups varied in size with 
seating for about twelve to about 75 depending upon the perceived need 
of the various organizations. 

CADAM/Lockheed designed several categories of training areas: 
(a) Lecture areas having built-in TV projection equipment (for 
motion visuals via video and for displaying computer screens to an 
entire audience) and overhead projectors built down into a portable 
desk/lectern. The students have work tables; they like the dark oak 
colors better than the white-topped tables because the white tables 
show dirt too well. The students, however, say they like the white 
tables; they seem to be easier on their eyes (perhaps because there 
was less contrast between their papers and the desk-tops) and easier 
to see their pencils, etc. The instructors liked the white marking 
boards w/ erasable color markers; next time they said they intend to 
construct the entire walls of the marking material ~ perhaps not 
white, but a light shade. This is typically the first room the 
students will meet in as they work on introductory cognitive or theory 
related objectives. 

(b) Classrooms for group hands-on lecture/practice area where the 
classroom is like the one described above, but the set-up includes the 
equipment the students will be training upon (CAD computer-terminals 
in this case) built into the instructional area. This is the second 
instructional areas the students normally use as they are taught the 
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procedure-related objectives. 



(c) "JbaD" areas wneif e uie e^i^iueaL ia aow-u^ «* *. ~<s- 
looking environment; like the ones where the equipment would normally 
be used. The instructor in these labs supervises about five students 
at a time. 

(d) "Classrooms" where their training products are "sold" to 
potential clients. These classrooms are much like the first environ- 
ment described above, but the colors are more active (bright blues, 
greens, reds) and they have large comfortable chairs and a relatively 
small amount of desk space. It seems to me CADAM has made excellent 
decisions about the design of the spaces — each related to identified 
cognitive objectives and affective goals. 

As suggested above, in classrooms designed for grouped instruc- 
tion, most of the chalk-boards in the classrooms I observed have been 
replaced by white or colored marking boards for "dry markers." Not 
only do the marking boards look more uodern, but their use does* not 
result in chalk dust which can seriously damage the computer disk 
drives. One organization said they chose lignt tan colored dry boards 
and tan for the walls and the desk tops. The result is an environment 
that looks somewhat drab. I was told "We do not want the classroom to 
look 'hi tech 1 . The white paper in workbooks, and texts will thus be 
somewhat brighter than the background, (desk, walls, etc.) and keep 
the students' attention on their learning task." 

In order to illustrate how the training program was helped by the 
new training facilities, I was taken twice to the still existing old 
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buildings (some of which were still in use) . Newly designed facili- 
ties had these characteristics: air conditioned, increase light quan- 
tity and quality, more attractive classrccm/ccrridor/cffice colors, 
more attention was paid to sound control (carpeting was typically 
included) , working (and often new) laboratory equipment. The newer 
environments seemed to have a calming and quieting effect on the 
learners. 

The modern facilities had extensive conduit networks allowing 
them to pull additional cables for electronic distribution of compuv- 
er-based or television-based materials. In the IBM example mentioned 
in the next paragraph the floor of the entire academic wing was built 
on a 15.5 cm raised floor with tail circuits in the classroom to 
permit convenient connection to the classroom's TV cameras and d3— "ay 
devices. A remote control box at the instructor's desk terminated 
conduit to the projection room to provide control of all equipment in 
that area (35mm & 16 mm projectors, sound amplifiers, video and audio 
cassette decks, equalizers, mixers, switchers, patch panels, video and 
audio cassette monitors, intercom). The conduit allowed video record- 
ings to originate ir all rooms and be recorded on any of the 24 VCRs 
located in the video control center. The conduit make it safely 
possible for them to use this technology. 

The flexible conduit system at an IBM training facility in Thorn- 
wood, N.Y. was a major reason for this facility being identified as 
the outstanding instructional facility in the United States (December 
1986 issue of Engineerin g Educator). The facility had two major 
design objectives: connectivity and flexibility. By "connectivity" 
they meant that everything is connected to everything else. The three 
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major networks include (a) baseband (to provide switchable quality 
video and audio signals) , (b) broadband (to provide video [commercial 
& public TV] & data [including electronic bulletin board communica- 
tion] to every location at the site) , & (c) establishment communica- 
tions link, ECL, (a token ring system to provide voice [telephone], 
data and video signals) . They are working on an all digital network 
..3DN (Integrated Systems Data Network). This connectivity results in 
flexibility. As the technology continues to develop, the systems will 
be condensed into one digital system which will be easier to use and 
be less costly to operate and maintain. The existing systems are used 
to broadcast instruction to IBM training facilities via satellite to 
the entire world. It allows for voice feedback to the transmitter 
site from the student's wherever they are located. 

Electronic feedback systems, built into individual student desks, 
have almost disappeared — at least in the classrooms visited in this 
survey. One ex-user of an electronic feedback system said: "They are 
time consuming and difficult to program and then yield little addi- 
tional useful information that is not available by direct class obser- 
vation — they simply are not useful." Another instructor, in an 
airline company classroom with electronic feedback devices said: 
"After you take class time to get the information, what do you do with 
it? Do you repeat the instruction until all students indicate they 
understand the instruction?" One executive meeting room I visited had 
a good use for an electronic feedback system; the switches were mount- 
ed beneath the table which allowed the participants to anonymously 
vote on issues. 

15 
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Many of the training organization were concerned about maintain- 
ing the security of the information they were teaching. Therefore, 
they often designed windowiess training buildings and prohibited their 
instructors from using "wireless transmitter- . " At these organiza- 
tions paper shredders or other security measures were taken to protect 
paper notes and materials. 

Summary 

Instructional development practices in outstanding U.S. industri- 
al and military organizations reveal many trends. Front-end analysis, 
for example, was uniformly stressed as a requirement for designing 
and developing instructional materials by managers, but instructional 
developers and instructors suggested that time is not always devoted 
to performing a thorough problem analysis. 

With regard to instructional design, most industrial and military 
training organizations had ISD models they generally followed. Espe- 
cially complicated presentations, or those for clients, were more 
likely to follow the rigorous instructional development procedure 
required by the development model than less complicated training 
programs for in-house use. Many of the training organizations are 
gradually moving from grouped instruction toward individualized in- 
struction. The use of interactive video disks, and of computer-based 
instruction frequently leads trainers away from totally grouped activ- 
ities. An increasing number of training organizations are designing 
group-oriented "enriched classrooms," or simulated work environments, 
in their classrooms. Interest in computer-based simulators is in- 
creasing. 
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Most of the organizations are using instructional technology 
extensively. CAI is being evaluated at many of the organizations but 
few are using it for more than perhaps five percent of their total 
instruction. Many organizations said CAI is costly to develop, but 
that the reduced instructional time required by most CAI programs, and 
the ability to distribute it easily over a number of locations, made 
it a potentially powerful medium. ICAI, CAI materials developing 
using "artificial intelligence" algorithms, was not being implemented 
by any of these organizations. Many of the visited organizations are 
adopting interactive video disk technology and almost all of them are 
considering it for near future instruction. Instructional television 
is used by almost all organizations. Dissemination of courseware via 
satellite or microwave facilities is a common practice. 

Many, but not all, of the successful program managers reported to 
a senior Vice-President or other high level administrator. These 
successful training managers had a great deal of support from their 
"line" administrator. Another characteristic of a successful instruc- 
tional development organization is that they were funded at the 
corporate or division level — and not by individual using depart- 
ments. The development of training materials is usually performed by 
a team located in a central production facility. 

There were also some evident trends involving the evaluation and 
revision of training materials and programs. Almost all instruction 
is evaluated. It appears, however, that instructors are seldom held 
accountable for their classroom instruction; pay increases and promo- 
tions are seldom related to classroom evaluations. Materials and 
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program evaluation (formative and summative) , however, are not meas- 
ured as consistently as classroom activities. 

Training facilities were usually specialized and used for specif- 
ic types of instructional objectives and clients (internal training or 
customer-support) . More facilities are being designed for individual- 
ized and computer-based instruction. Most organizations had multi- 
media classrooms which were used extensively to supplement, or re- 
place, some group-o-,iented lectures. Light-colored marking boards 
have replaced chalk boards in most instructional facilities; the chalk 
dust is hard on computer components and on general classroom mainte- 
nance. The modern facilities had extensive conduit networks which 
allows them to install additional fiber or conventional cables for 
electronic distribution of computer-based or television-based materi- 
als. Electronic feedback systems have almost disappeared from corpo- 
rate classrooms. Facilities are increasingly being built with securi- 
ty in mind. Thus, windowless classrooms are increasingly common and 
there is concern about how to keep audio or electronic emanations from 
being picked up outside the training facilities. 

Many trends in the best adult training organizations in the 
United States became evident in this study. Some instructional tech- 
nologies are becoming reliable and highly used instructional tools. 
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